Evaluation Methodologies for Interactive Speech Systems
نویسنده
چکیده
In this paper, several criteria and paradigms are described to measure the performance of spoken language systems. The focus is on the evaluation of natural language understanding components. These evaluations are carried out in the domain of spontaneous human-human interaction as supported by automatic translation systems. They are also applied in the domain of spontaneous human-machine interaction typically used in information retrieval applications. Some system response evaluation paradigms for different applications and domains are discussed in more detail. It is also shown that official performance tests and site-specific evaluation paradigms are complementary in use.
منابع مشابه
A Database for Automatic Persian Speech Emotion Recognition: Collection, Processing and Evaluation
Abstract Recent developments in robotics automation have motivated researchers to improve the efficiency of interactive systems by making a natural man-machine interaction. Since speech is the most popular method of communication, recognizing human emotions from speech signal becomes a challenging research topic known as Speech Emotion Recognition (SER). In this study, we propose a Persian em...
متن کاملConsiderations in the design and evaluation of spoken language dialog systems
In this paper we summarize our experience at LIMSI in the design, development and evaluation of spoken language dialog systems for information retrieval tasks. This work has been for the most part carried out in the context of several European and international projects. Evaluation plays an integral role in the development of spoken language dialog systems. While there are commonly used measure...
متن کاملA New Statistical Model for Evaluation Interactive Question Answering Systems Using Regression
The development of computer systems and extensive use of information technology in the everyday life of people have just made it more and more important for them to make quick access to information that has received great importance. Increasing the volume of information makes it difficult to manage or control. Thus, some instruments need to be provided to use this information. The QA system is ...
متن کاملTowards Innovative Evaluation Methodologies for Speech Translation
This paper reports on activities within the C-STAR1 consortium which aim at novel speech translation technologies and their evaluation. In C-STAR, current state-of-the-art speech translation systems developed by the partners are evaluated and discussed on a regular basis by means of evaluation workshops. The objectives of these workshops are to provide a framework for the validation of existing...
متن کاملDesigning interactive speech systems - from first ideas to user testing
Springer. Paperback. Book Condition: New. Paperback. 278 pages. Dimensions: 9.2in. x 6.1in. x 0.8in.Designing Interactive Speech Systems describes the design and implementation of spoken language dialogue within the context of SLDS (spoken language dialogue systems) development. Using an applications-oriented SLDS developed through the Danish Dialogue project, the authors describe the complete ...
متن کامل